Boosting SMT Performance by Speculation Control

Simultaneous Multithreading (SMT) is a technique that permits multiple threads to execute in parallel within a single processor. Usually, an SMT processor uses shared instruction queues to collect instructions from the different threads. Hence, an SMT processor’s performance depends on how the instruction fetch unit fills these instruction queues. On each cycle the fetch unit must judiciously decide which threads to fetch instructions from. This paper proposes a new instruction fetch scheme that uses both fetch prioritizing and fetch gating for SMT processors. Fetch prioritizing sets up fetch priority for each thread based on the number of unresolved low-confidence branches from the thread, while fetch gating prevents fetching from a thread once it has a stipulated number of outstanding low-confidence branches. Based on the fetch priority of each thread, our fetch scheme finds threads that are most likely to be in their correct paths. This improves the overall throughput of an SMT processor by reducing the number of wrong-path instructions in the pipeline. Our experimental evaluation shows that, on the average, our fetch scheme provides 14.9% speedup over ICOUNT, which is the best fetch policy reported so far for SMT.